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© Image keying generator for video special effects, 

© An image processing system operates on first 
and second image data (V and Bg) to produce 
data defining a combined image. The first im- 
age data (V) represents a keyed image compris- 
ing a first object (F) keyed over a portion of a 
first background (B) in accordance with keying 
data (K)- The second image data (Bg) represents 
a background. The system is arranged so that 
the second image data (Bg) is weighted by 
respective control data (<x) related to said key- 
ing data (K) before being combined with the 
first image data (V). In another embodiment the 
first image data represents a second object 
independently keyed over the keyed image in 
accordance with second keying data and the 
control data is further related to the second 
keying data. In a further embodiment the first 
image data represents an image derived by 
cross fading between the first keyed image and 
a second keyed image. 
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This invention relates to video image keying systems and methods. 

Keying systems are widely used for picture editing and composition in television and are used to key a fore- 
ground image Into a background image. For example a keying system may be used for keying letters or other 
characters forming a caption into a picture to provide information relating to the subject matter of the picture. 

5 In such a case video signals representing the characters, as selected by the operator, are generated by a source 
such as a character generator, for example the character generator sold by us under the trade mark "CYPHER", 
When a character is selected, the generator not only provides the video signals representing the character 
which may be regarded as foreground signals F, but it also provides internal control signals K which are used 
to key the character or foreground video signals over other video signals representing a background matte B, 

10 Normally the background matte B is a uniform black. The resulting combined signal is output from the source 
for further processing by other television image processing apparatus. The control signals K are also used with in 
the source to key video signals representing white (value 1) over video signals representing black (value 0) to 
produce a key signal a which is output from thesource for external use together with the combined signal. Thus, 
Hie output of the character generator comprises the video signal representation V of the foreground F keyed 

16 over the background B together with the external key a. The external key a is used subsequently in external 
circuitry as will be described in greater detail hereinafter. 

At present it is usual to perform the keying of the foreground F over the background matte B within the 
source character generator using a linear interpolating circuit in which the internal control signals K function 
as an interpolating co-efficient. The video V which is the result of the interpolation can then be represented by 

20 V = F.K + 8(1 - K) (1) 

it is also usual for the control signals K to have a maximum value, which may be 1 or less than 1, for pixels 
lying within the character represented by the video signals F and to have value 0 for pixels within the background 
matte 8, except fora zone round the boundary of the character over which the control signals K decline gradually 
from the maximum value, eg 1 , to 0 to produce a soft edge between the character F and the background matte 

25 B. As is well known, this so-called soft edged keying is employed to reduce aliasing which would otherwise 
arise between character and background. The maximum value of K will be less than 1 where a foreground image 
F is to be combined transparently with the background matte B. Thus the coefficient K in the above expression 
for V may have values in the range: 

0 23 K 285 i 

30 and it will be understood that similar soft edging will also appear in the external key signal a. In the accompa- 
nying drawings Figures 1 (A) and 1 (B) together illustrate a character generator or source of the above described 
form, conditioned to generate an output video as represented by the above expression for V and also to gen- 
erate the above mentioned external key a. All the Figures in the drawings are in the form of schematic functional 
diagrams in which different circuits elements are represented by blocks identified by functional symbols denot- 

35 ing the nature of the respective circuit elements. Specific explanation of individual functional units in the draw- 
ings is considered to be unnecessary for an understanding of the invention by those possessed of the appro- 
priate skills. 

Returning to Figure 1(A) it will be appreciated that where K = 0 pixels in the video image V will relate solely 
to the matte background B, where K = 1 pixels in the video image V will relate solely to the foreground character 

40 F and that for pixels within the soft edges, where K is between 1 and 0, the output video V will include a con- 
tribution both from F and from the background B. In the special case where the background matte is black of 
value 0, then the term B(1-K) will remain 0 regardless of the value of K r but the term F.K will diminish with K 
at the soft edges, thereby "darkening" the character, in effect increasing the contribution from black. It will also 
be appreciated from Figure 1(B) that the value of a will track that of K, being non-zero where K is non-zero. 

45 The external key a is used by externa! circuitry to key the video signal V output from the source into video 

signals representing a picture which may be regarded as arbitrary external background Bg. in this way a re- 
sultant picture R is produced comprising the background picture Bg and the foreground character F. The ex- 
ternal keying of the video V into the arbitrary external background Bg has hitherto usually been performed by 
another linear interpolation process as illustrated in Figure 1(c) to produce the result: 

50 R = a V + Bg(1 - a) 

Bearing in mind that from equation (1) hereinabove V - F.K + B(1 + K) then the above equation for R shows 
that any contribution of the original background matte B in the signal V is multiplied by a. As mentioned here- 
inabove pixels within the soft edges are darkened towards black by the keying performed within the character 
source. In other words pixels in the soft edge output from the character source include an inseparable contri- 

55 button from the black background of the matte. Therefore, the result signal R from the external interpolator in- 
cludes the black background contribution in the soft edges thereof in addition to the external background Bg 
and this black contribution is therefore a spurious signal which may be seen as halos or shadows in the result 
image R. Whilst these unwanted artefacts are usually feint they are nevertheless visible and can be disturbing. 
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Similar problems can occur in other systems where two or more keying operations are required over the same 
area of a picture, for example in systems such as described in our British Patent No. 2,113,950 (equivalent 
USA patent 4,602,286), or in systems in which different foreground objects can be internally keyed over two 
copies of the same background picture and an output produced by internally cross fading between one stamped 
5 copy and the other 

The object of the present invention is to reduce the problem of spurious signals such as illustrated above. 
According to one aspect of the present invention there is provided an image processing system in which first 
image data, representing at least one foreground image keyed to a portion of a first background in accordance 
with respective keying data, is combined with second image data, representing a second background, to pro- 
10 duce data defining a combined image comprising said at least one foreground image keyed to a portion of said 
second background, the system being arranged such that the second image data is weighted by controi data 
derived from said keying data before being combined with the first image data such that data representing the 
first background is replaced by the second image data. 

According to another aspect of the invention there is provided a keying system for television image proo- 
fs essing, the system comprising: a source of first video signals V representing a foreground object F keyed over 
a first background B; a source of second video signals representing an arbitrary background Bg; a source of 
key signals (cx) for keying the foreground object over the arbitrary background Bg; and a keying circuit respon- 
sive to said first and second video signals and said key signals to produce the result = V + Bg(1 - a). 

On the condition that> 

20 

V *» FK 4- B(l-K) as above (1) 
B = 0; and 

25 

= K, (2) 

the above results R is: 

R = FK + Bg{1 - K) 

30 which yields the required blend for soft edging between the foreground and the arbitrary background, Bg, with 
no contribution in the Result from the background B. That is to say, the first background B is replaced by the 
new arbitrary background Bg in such a way as to avoid the introduction of spurious artefacts such as halos or 
shadows in the resultant image. 

According to another aspect of the invention there is provided an image processing system comprising a 

35 first data source for supplying first image data representing a first image formed from at least one foreground 
image keyed to a first background image by interpolation in accordance with at least one respective first inter- 
polation co-efficient and for supplying a second interpolation coefficient derived from the or each first interpo- 
lation coefficient, and a divider for dividing the first image data by the second interpolation co-efficient 

In a further aspect the invention provides a method of processing image data, the method comprising sup- 

40 plying first image data representing a first image formed from first image data representing at least one fore- 
ground image keyed to a first background image by interpolation in accordance with at least one respective 
first interpolation co-efficient, deriving a second interpolation co-efficient from the or each first interpolation co- 
efficient, dividing the first image data by the second interpolation co-efficient. The above and further features 
of the invention are set forth with particularity in the appended claims and together with advantages thereof 

45 will become clearer from consideration of the following detailed description of exemplary embodiments of the 
invention given with reference to the accompanying drawings. 
In the drawings: 

Figure 1 illustrates a simple character generator at (a) and (b) and a prior art external keying process at 
(c) as previously discussed hereinabove, 
so Figure 2 illustrates a simple external keying system according to the present invention; 

Figure 3 illustrates a system including an external keyer for keying overlapped foreground objects onto the 
same background; 

Figure 4 illustrates a system including an external keyer for keying two different foreground objects over 
respective copies of the same background and for cross fading between the two; and 
55 Figure 5 illustrates a system in which a foreground image is pre-processed before being delivered externally 

to a conventional keying circuit 

Further reference will now be made to Figures 2 to 5 of the accompanying drawings, which illustrate several 
different exemplary embodiments of the invention. Turning first to Figure 2(A), there is illustrated a system in- 
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eluding a simple external or so-called downstream keying circuit according to the invention. The keying circuit 
of Figure 2(A) operates on the video V output from the character source of Figure 1 (A) under control of the key 
a output from the source shown in Figure 1(B). There is a second video input in Figure 2(A) namely a picture 
which can be regarded as an externa! arbitrary background Bg. It will be noted that in the downstream keying 
circuit of Figure 2(A) there is no multiplying circuit in the input path for V, as there wouid be if the keying circuit 
were a linear interpolating circuit such as used hitherto and illustrated in Figure 1(c) of the accompanying draw- 
ings. The Result which is output from the Figure 2(A) downstream keyer is given by the equation: 

R = V + Bg (1 - a) 

but 

V - F.K + B(1 - K). (1) 
and if B = 0 in equation (1), then substituting for V and a gives 

R = FK + Bg(1 - K). 

That is to say, the result R output from the external keyer circuit of Figure 2(A) is the foreground F keyed 
onto the external background Bg with no contribution from the original background B, which is the desired result. 
The condition that B = 0 implies that the 'Internal" background B in the video represented by the first video 

V signal is black matte. However, the CCIR specification for broadcast television signals specifies a luminance 
value 16 for black, and the second video signals representing the "external " arbitrary background might have 
biack equal to sixteen in the scale of luminance signal values, In this event as illustrated in Figure 2 (B) of the 
accompanying drawings the black "offset" value of 16 would, in practice, be subtracted from both video signals 

V and Bg before applying them to the keying circuit, and would then be re-added to the Result The circuit of 
Figure 2(B) can be simplified to the circuit of Figure 2 (C). 

The invention would usually be applied to colour television signals in digital form comprising different chan- 
nels for different colour components eg YUV or RGB. In this specification the description is confined to one 
channel, assumed to be the Y channel of a YUV system. For such a system the Y component to represent black 
has the value 16 (in the scale 0-255)* as already discussed. For U and V components, the CCIR specification 
prescribes the value 128 (again on the scale 0 to 255) for zero chroma, and appropriate measures for dealing 
with the offset are required in the U and V channels also. Similar considerations apply to the generation of the 
external keying signal a from the internal key signals K. 

Figures 3(A), 3(B) and 3(C) illustrate a system for keying overlapping foreground objects F1 and F2 onto 
the same background B, followed by subsequent keying of the output over an external arbitrary background 
Bg. Figure 3(A) shows the circuit components for keying F1 and F2 and B, Figure 3(B) shows the circuit com- 
ponents for generating the external key a, while Figure 3(C) shows the external keying circuit. The video output 
from Figure 3(A) is represented by:- 

V - [Fi K a + BCl-Ki) ] (1~K 2 ) 4- F 2 K a 

- B<i-KJ (1-K 2 ) + F x K x (1-K Z ) + F 2 K 2 . (3) 

The key signal output from Figure 3(B) is represented by:- a = Ki (1 - K 2 ) + K 2 . 

It will be noted that Ki is the only output from the first of the two multiplying stages in Figure 3(B) because the 
video signal for black is 0. It will also be noted that the external or downstream keyer circuit of Figure 3(C) is 
exactly the same as that shown in Figure 2(A) and that similar considerations regarding the offset values of 
biack as discussed hereinabove in relation to Figures 2(B) and (C). Since the Figure 3(C) downstream keyer 
is the same as that in Figure 2(A) it follows that the result R of the external keying operation performed in the 
Figure 3(C) circuit is again represented by the equation: 

R = V + Bg(1 - a). 

If B = 0 in equation (3) then substituting: 



--2 HP 


Kx 


(1 


-K 2 ) 




F 2 


K 2 


+ 


Bg 


(1[K 1 (X-K 2 )+K 2 ]> 


= F 1 


Kx 


(1 


-K 2 ) 


+ 


Fa 


K 2 




Bg 


(l-Ki-Ka + Kj. K 2 ) 


- F x 


Ki 


(1- 


-K 2 ) 


+ 


F 2 


K 2 


+ 


Bg 


(1-K X ) (1-K 2 ) . 



It should be noted that Bg (1-K^) (1-K 2 ) is the same term as was applied to B in equation (3) and is therefore 
free from contributions from the original background B, as required. 
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The arrangement illustrated by Figures 3(A), 3(B) and 3(C) can be used in an image composition system 
such as shown in our British Patent 2,113,950 and USA Patent 4,602,286. 

The group of figures comprising Figure 4 illustrate an arrangement for keying two different foreground ob- 
jects Ft and F 2 over two respective copies of the same background B in which arrangement the two results are 

5 cross faded, as are the two internal keys. Subsequently the cross faded foregrounds F 1 and F 2 are keyed by 
a downstream keyeer into an external arbitrary background Bg. The initial background B is again assumed to 
be a black matte and thus to have a value notionally of 0, As an illustration of the use to which the form of the 
invention may be put, the foreground objects F1 and F2 may be symbols denoting different sports, for example 
a cricket bat and a tennis racket keyed into the top left hand corner of a black matte. The external background 

10 Bg may be a sports commentary of a cricket match and a tennis match, with the commentator switching at will 
between to two. At any one time the appropriate symbol is keyed into the external background Bg, and when 
the commentator switches from one match to the other, the two symbols F1 and F2 are cross faded. As rep- 
resented in Figure 4(A), cross fading of the foregrounds F1 and F2 can be affected under contro! of the operator 
by a dissolve signal D, which determines the "profile" of the fade. The Figure 4 arrangement provides an output 

15 video V, whilst Figure 4(B) shows the circuit arrangement for producing the corresponding external key a de- 
rived from the two internal keys K t and K 2 , The expressions for the signals V and a are given by the following 
mathematics: 

w V = [Fi K x + BU-KJ] D + [F 2 K z + B(l-K 2 ) ] (l-D) 

- B[D(K a -Ki) + (1-K 2 )] + F l K x D + F 2 K 2 (l-D) (5) 

a = Ki D + K 2 (1 - D) (6) 

25 Subsequently V and a are applied to an external downstream keying circuit as illustrated in Figure 4(C). 

Again it should be noted thatthe circuit of Figure 4{C) is identical to that of Figure 2(A) and similar considerations 
regarding offset values apply. When V and a are applied to the external keying circuit shown in Figure 4(C) 
together with the external background Bg, the following mathematics expresses the result R output therefrom. 
From equation (6) 

30 

(1 - ~) 1 - (K x D + K 2 (l-D)) 

= 1 - Kj D- K 2 + K 2 D 
35 - D(K 2 - K L ) + (1-K 2 ) (7) 

If B is 0 in equation (5) then 

V = Fi K 1 D + F 2 K 2 (1 - D) 
40 Now R = V + BG (1 - a), and so substituting both for V (with B - 0) and for (1 - a) gives: 

Result = Bg[D(K 2 -K,) + (1 - K 2 ) ] + F^D + F 2 K 2 (1 - D) 
This is the same term for Bg as was applied to B in equation (5) and is therefore the required result, That 
is to say, in the result R the original internal background B is replaced with the external background Bg with 
no residual contribution from the original background. 
45 In each of Figures 2, 3 and 4 various video signals V, which have previously been created by keying one 

or more foreground signals F onto a background matte signal B, are supplied to a downstream keyer together 
with an external keying signal a derived from the internal keying signals K used to produce the video signal. 
The derived external keying signal a is used to weight an arbitrary background signal Bg in the downstream 
keyer and the weighted background signal is then combined with the video signal V, unweighted, to produce 
50 the desired result. 

An alternative arrangement alfows a conventional keyer similar to that shown in Figure 1(C) of the accom- 
panying drawings to be used. The keyer shown in Figure 1(C) is a widely available apparatus and by using 
such a keyer the costs of the system can be kept to a reasonable level. The alternative arrangement is shown 
in Figure 5 of the accompanying drawings and comprises a conventional source, such as a CYPHER character 
55 generator which provides a video signal V and a keying signal a similar to those signals already discussed here- 
inabove in relation to Figures 3 and 4 of the drawings, and a conventional keyer. The video signal comprises 
a foreground F keyed onto a black matte background B by a key K. The output video V and the internally derived 
external key a are output to a look up table LUT which divides V by a. In setting up the look up table consid- 
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eration must be given to the black offset in the foreground data F and the key data (see Figures 2B and 2C). 
The output from the look up table is delivered together with the external key a and an arbitrary background Bg 
to the conventional keyer which keys the a-divided video V onto the background using the key a. It will be noted 
that net effect of the look up table is to precondition the foreground signal F such that rt is supplied unchanged 

5 to the conventional keyer. Once inside the keyer the foreground signal is multiplied by the external key a and 
added to the background weighted by (1-a) to produce the result R = V + Bg{1 - a) as before. Thus, the look 
up table performs the inverse operation to that performed by the multipliers within the source. Thus, the look 
up table, which may be a stand-alone unit but preferably is built into the source, serves to pre-compensate the 
background video V so that it is not processed for a second time by the same key data in the conventional keyer 

10 thereby ensuring that the background matte B is replaced completely by the external background Bg in the con- 
ventional keyer without the introduction of unwanted artefacts. 

In each of the above described embodiments the video signal is created by keying one or two foreground 
signals over a black matte background using a respective internal key for the or each foreground signal. An 
external key signal is derived from the internal keys and is subsequently used to combine the video signal with 

is an external background signal of arbitrary content. The embodiments are not limited to the use of a video signal 
created from only one or two foreground signals and can be applied equally to a video signal formed from mul- 
tiple foreground images so long as the keying signal for each foreground signal is available for use in deriving 
the external keying signal. 

Having thus described the present invention by reference to preferred embodiments it is to be well under- 

20 stood that the embodiments in question are exemplary only and that modifications and variations such as will 
occur to those possessed of appropriate knowledge and skills may be made without departure from the spirit 
and scope of the invention as set forth in the appended claims and equivalents thereof. 



25 Claims 



1. An image processing system in which first image data, representing at least one foreground image keyed 
to a portion of a first background in accordance with respective keying data, is combined with second image 
data, representing a second background, to produce data defining a combined image comprising said at 
30 least one foreground image keyed to a portion of said second background, the system being arranged 

such that the second image data is weighted by control data derived from said keying data before being 
combined with the first image data such that data representing the first background is replaced by the sec- 
ond image data. 

3$ 2. An image processing system as claimed in claim 1, wherein said first image data represents plural fore- 
ground images independently keyed to portions of said first background in accordance with respective plu- 
ral keying data, and wherein said control data is derived from said plural keying data. 

3. An image processing system as claimed in claim 1 or 2 wherein the control data is derived by operating 
40 with said keying data on data representing image white and image black. 

4. An image processing system as claimed in any preceding claim, wherein said first background is a black 
matte. 



5, An image processing system as claimed in any preceding claim, wherein said first and second image data 
45 are combined by addition. 

6. An image processing system as claimed in any preceding claim, wherein said foreground and background 
images each comprise a plurality of picture elements and are combined on an element by element basis. 

so 7. An image processing system as claimed in any preceding claim, wherein the combined image data is out- 
put for display. 

8* A keying system for image processing, the system comprising; 

a source of first video signals V representing at least one foreground object F keyed over a first background 

a source of second video signals representing an arbitrary background Bg; 

a source of key signals (a) for keying the foreground object over the arbitrary background Bg; and 

a keying circuit responsive to said first and second video signals and said key signals to produce the result 

6 
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- V + Bg{1-a). 

9, An image processing system comprising a first data source for supplying first image data representing a 
first image formed from at feast one foreground image keyed to a first background image by interpolation 
in accordance with at least one respective first interpolation co-efficient and for supplying a second inter- 
polation coefficient derived from the or each first interpolation coefficient, and a divider for dividing the first 
image data by the second interpolation co-efficient. 

10, An image processing system as claimed in claim 9, further comprising a second data source for supplying 
second image data representing a second background image, and a keying circuit for keying said divided 
first image data and said second image data by interpolation in accordance with the second interpolation 
co-efficient 

11, An image processing system as claimed in claim 9 or 10, wherein said first background image is a black 
matte. 

12, An image processing system as claimed in claim 9 or 10 or 1 1 , wherein the second interpolation coefficient 
is derived by interpolating between signals representing image black and image white in accordance with 
the or each said respective first interpolation co-efficient. 

13, An image processing system as claimed in any of claims 9 to 12, wherein the divider comprises a look- 
up table responsive to the first image data and said second interpolation coefficient for outputting data 
corresponding to said divided first image data. 

14* An image processing system as claimed in any of claims 10 to 13, wherein said first and second images 
each comprise a plurality of picture elements and said first and second data are keyed on a element by 
element basis. 

15. An image processing system as claimed in claim 14, wherein said at least one first and s^ld second in- 
terpolation co-efficient are each defined on an element by element basis, 

16. A method of processing image data, the method comprising supplying first image data representing a first 
image formed from first image data representing at least one foreground image keyed to a first background 
image by interpolation in accordance with at least one respective first interpolation coefficient, deriving a 
second interpolation coefficient from the or each first interpolation coefficient, dividing the first image data 
by the second interpolation co-efficient 

17. A method as claimed in claim 16, further comprising obtaining second image data representing a second 
background image, and keying said divided first image data and said second data by interpolation in ac- 
cordance with the second interpolation co-efficient. 

18. A method as claimed in claim 16 or 17, wherein data in a look-up table is used to determine said divided 
first data from said first data. 
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© Image keying generator for video special effects. 

(57) An image processing system operates on first 
and second image data (V and Bg) to produce 
data defining a combined image 
(R»V+Bg(1-a)). The first image data (V) repre- 
sents a keyed image (V=FK+B/1-K)) comprising 
a first object (F) keyed over a portion of a first 
background <B) in accordance with keying data 
(K). The second image data (Bg) represents a 
background. The system is arranged so that the 
second image data (Bg) is weighted by respec- 
tive control data (a) (flg.18) related to said 
keying data (K) before being combined with the 
first image data (V). In another embodiment 
(fig.3) the first image data (V) represents a 
second object independently keyed over the 
keyed image in accordance with second keying 
data (K2) and the control data {*) is further 
related to the second keying data. In a further 
embodiment (ftg.4) the first image data repre- 
sents an image derived by cross fading between 
the first keyed image and a second keyed im- 
age. 
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